patterns and variations in native and non-native interlanguage pragmatic rating: effects of rater training, intercultural proficiency, and self-assessment

although there are studies on pragmatic assessment, to date, literature has been almost silent about native and non-native english raters’ criteria for the assessment of efl learners’ pragmatic performance. focusing on this topic, this study pursued four purposes. the first one was to find criteria for rating the speech acts of apology and refusal in l2 by native and non-native english teachers and to examine if apology and refusal situations make any variations in the criteria applied to the rating of l2 apology and refusal discourse completion tasks (dcts). the second was to discover the interrelationship of the non-native raters’ self-assessment of their pragmatic competence and their accuracy of rating the two speech acts against native benchmark. the third was to explore the interrelationship of the non-native raters’ intercultural proficiency and their accuracy of rating the two speech acts against native benchmark. the final purpose of this study was to uncover the possible effect of the training session on the accuracy of non-native raters’ ratings against native benchmark. to this end, 12 different pragmatic situations for apology and refusal dcts accompanied by an l2 learner’s response to each situation were rated by 50 native and 50 non-native english teacher raters. besides rating, the raters were asked to write their criteria for rating the response to each dct situation.the content analysis of raters’ comments revealed five apology and eleven refusal criteria. the descriptive statistics for apology and refusal rating indicated variations across situations. then correlations between non-native raters’ self-assessment of their pragmatic competence and their accuracy of speech act ratings as well as between their ratings and their intercultural proficiency revealed that there were strong relationships among them. a number of t-tests and chi-squares were computed to see the effect of the training session. the results indicate that non-native raters approached native benchmark in terms of rating and criteria after the training session. this study bears evidence to the necessity of rater training and informed, rubric-based pragmatic rating.